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Introduction 


Teachers strive to deliver specialized and individualized instruction to address the needs of 
all students. To differentiate instruction, educators are turning more frequently to tech- 
nology-based interventions designed to increase engagement, individuate instruction, and 
provide efficient, embedded progress monitoring. Unfortunately, the efficacy of technol- 
ogy-based interventions is inconclusive and sometimes contradictory (Young et al. 2012). 
Inconsistent implementation may be one explanation for these ambiguous findings 
(MacDonald 2014; Valdez 2005). Like traditional interventions, technology-based inter- 
ventions are intended for use in specific contexts under particular conditions. However, the 
extent to which technology-based interventions are implemented as intended remains 
largely unexplored. This manuscript provides an evaluation of qualitative and quantitative 
data generated in two studies of KinderTek (KTEK), an iPad-delivered kindergarten 
mathematics intervention, to determine the relationship between instructor-reported 
experience with technology and intervention implementation. 


Supporting early mathematics development 


One area in which technology-based interventions could have a significant impact is in the 
area of early mathematics. Longitudinal studies of mathematics achievement indicate 
kindergarten mathematics to be foundational to future mathematics performance (Morgan 
et al. 2009), and mathematics difficulties in the late elementary grades can often be traced 
back as early as kindergarten (Bodovski and Farkas 2007; Duncan et al. 2007; Hanich et al. 
2001). Accordingly, efforts to create effective early mathematics interventions targeting 
the development of whole number understanding and proficiency (Jordan et al. 2012) are 
crucial to ensure students have a foundation on which to build an increasingly robust and 
complex understanding of mathematics. These targeted interventions provide intensive 
instruction to students whose academic achievement levels require intervention above and 
beyond core classroom mathematics instruction (Fuchs et al. 2012). Encouragingly, recent 
research reports suggest that a number of technology-based early mathematics interven- 
tions show promise for improving mathematics achievement for preschool, kindergarten, 
and early elementary students (Fien et al. 2016; Foster et al. 2016; Schacter et al. 2016) 
However, schools continue to encounter resource and time obstacles to delivering indi- 
vidualized, targeted intervention instruction, particularly in the area of early mathematics 
(Strand Cary et al. 2015). 


Educational technology use 


Technology-based interventions can mitigate resource and time challenges, particularly if 
the interventions customize instruction to individual students’ needs (Taylor 2004). If 
devices, software, and applications are affordable and compatible with classroom contexts 
(Keengwe et al. 2012), such interventions are potential answers to the intervention 
implementation dilemma in resource-strapped contexts (U.S. Department of Education 
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Office of Educational Technology 2016). However, questions remain about the extent to 
which educational technology can reliably improve student outcomes (Slavin and Lake 
2008; Young et al. 2012), and this lack of efficacy evidence is problematic. 

Recent survey results suggest technology use is quite prolific in US schools (Gray et al. 
2010), but not clearly linked to student gains. Most contemporary classrooms involve 
either teacher or student use of technology, and teachers report technology has positive 
effects on teaching and student engagement (DigEdu 2014). Additionally, mobile tech- 
nologies such as iPads are ubiquitous in today’s schools, and nearly 40 % of all 2-4 year- 
olds have a working knowledge of iPads and tablets (Common Sense Media 2013). 
Educational technology is also common in the early grades and young children are using 
tablet-based apps to practice early academic skills. In fact, nearly 72 % of iTunes top 
selling apps are geared toward preschool and elementary-aged learners (Common Sense 
Media 2013), and researchers report that iPads can be used to improve young children’s 
learning (Kermani and Aldemir 2016; Ray 2015). Notably, however, fewer teachers report 
technology has strong effects on student achievement and learning outcomes, and teachers 
attribute these inconsistencies to a variety of technology-based (e.g., lack of hardware) and 
logistical (e.g., loss of class time) barriers (DigEdu 2014). Although students often use 
technology-based educational interventions independently, teachers, classroom structures, 
and school capacities are extremely important factors in intervention implementation. 


Issues in technology-based intervention implementation 


Addressing the current shortcoming of technology requires a focus on teacher facilitation 
and integration of technological materials (U.S. Department of Education, Office of 
Educational Technology 2016), which is crucial for successfully integrating technology- 
based interventions in authentic learning contexts (Hughes 2005). Educators’ training and 
prior experiences with technology can affect the integration of classroom technology 
(Matzen and Edmunds 2007) and create barriers to successful technology-based inter- 
vention implementation, resulting in limited implementation and student use (Palak and 
Walls 2009). Prior research has explored the impact of multiple types and levels of barriers 
to technology use (Ertmer 1999). First-order barriers include general teaching and school- 
based demands, as well as challenges related to technological functionality, training, and 
support. Second-order barriers include personal fears, disinterest, and distrust. 

Regardless of type, barriers to classroom technology use are surmountable. Successful 
classroom integration of technology-based interventions requires instructors to enact sys- 
tems supporting use of the intervention. For example, proper training and modeling of 
student-centered technology integration has increased the likelihood of teachers success- 
fully integrating technology in their classrooms (Matzen and Edmunds 2007). Factors such 
as previous experience, comfort with technology, and self-perceived barriers to technology 
use impact the extent to which teachers are able to integrate educational technology 
skillfully into traditional academic structures. Additionally, the consistent, standardized 
implementation of educational interventions is an important aspect of replicable research. 
The ability to compare the performance of treatment and control groups in development 
studies is dependent on a relatively high level of consistency within groups with regard to 
instructional exposure and treatment intensity (Durlak and DuPre 2008; McKenna et al. 
2014). Thus, teacher experience with technology and barriers to technology integration are 
important considerations for technology-based intervention development and implemen- 
tation, and the relationships between these factors and technology-based intervention 
implementation should be studied explicitly. 
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This two-study investigation explored whether instructor experience with technology, 
measured by iPad experience and self-reported barriers to technology integration impacted 
implementation, measured by time of student use of KTEK, an early mathematics inter- 
vention delivered through a system of apps that aims to assist students who struggle in 
math. Research questions were: (a) How does instructor experience with technology and 
KTEK intervention implementation vary in each study? (b) To what extent is instructor 
experience with technology related to the implementation of the KTEK intervention as 
measured by student use? (c) How are barriers to classroom technology use related to 
intervention implementation? and (d) To what extent are study | relationships replicated in 
study 2? 


Methods 


KTEK development began in the fall of 2011 and culminated with a pilot study conducted 
in the winter/spring of 2014. Study 1 was part of a feasibility study conducted during 
winter/spring of 2013 and Study 2 was part of the pilot study. All KTEK research efforts 
took place in one school district in the Pacific Northwest region of the United States. 
Approximately 17 % of the participating district’s students receive special education 
services. Within the three participating schools, 7-27 % of the total student population is 
reported to be English language learners, and school-wide free—and-reduced lunch eligi- 
bility ranged from 43 to 78 % (National Center for Educational Statistics- NCES 2013). In 
all, KTEK was implemented in 21 classes (Study 1 = 10; Study 2 = 11), a mix of morning 
and afternoon half-day kindergarten classes taught by thirteen teachers and six educational 
assistants (EAs). 


Participants: study 1 


Study | participants included one male and five female teachers with varying levels of 
teaching experience and technical expertise who taught a total of ten morning and after- 
noon kindergarten classes. On average, Study | educators had more than 15 years of 
teaching experience and 7 years of experience at the kindergarten level. All Study 1 
instructors were white. Participants implemented the intervention independently in small 
groups during center time in their classrooms to examine the feasibility of KTEK. Given 
the intervention’s primary goal of supporting learning for young students who struggle 
with math, researchers identified five potentially at-risk students from each class. Target 
students (n = 50) were selected based on teacher recommendation, evidence of consistent 
attendance, and potential mathematics risk as determined by screening data collected from 
student performance on a brief battery of number-sense, curriculum-based measures 
(ASPENS; Clarke et al. 2011). The Study 1 student sample was 52 % male, 40 % His- 
panic, 26 % English language learners, and 22 % received special education services. All 
participating students qualified for free or reduced price lunch. 


Participants: study 2 
Based on implementation difficulties noted in Study 1, EAs were recruited to assist with 


the implementation of KTEK in study 2. Thus, Study 2 participants included seven female 
teachers and six EAs (one male) with varying levels of educational attainment, teaching 
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experience, and technical expertise. Study 2 instructors had over 11 years of teaching 
experience and approximately 4 years of experience at the kindergarten level, on average. 
The participants implemented the intervention in structured pull out groups outside of the 
classroom to examine the potential promise of KTEK for use in a tiered intervention 
system. Because KTEK’s primary development goal was to support the learning of low 
performing kindergarten students, ten at-risk students from each class were identified as 
target students for the pilot study. As in study 1, target students were selected based on 
teacher recommendation, evidence of regular attendance, and screening data collected 
from student performance on a brief battery of number sense curriculum based measure- 
ments (Clarke et al. 2011), and then randomly assigned to condition. After attrition, the 
final analytic student sample for Study 2 was comprised of 45 treatment and 49 control 
participants. The Study 2 student sample was 58 % male, 12 % Hispanic, 6 % English 
language learners, and 14 % received special education services. Almost 40 % of the 
participating students qualified for free or reduced price lunch. 


Intervention 


KTEK uses the affordances of iPad technology to target whole number content outlined in 
the Counting & Cardinality, Operations & Algebraic Thinking, and Number & Operations in 
Base Ten strands of the Common Core standards (CCSSO 2014) for kindergarten. In contrast 
to practice apps intended solely for fluency building, the KTEK intervention system provides 
explicit instruction and targeted review appropriate to each student’s learning needs through 
two iPad apps. Students use the student app individually, while the teacher app allows real- 
time monitoring of student use and progress. KTEK customizes learning experiences by 
incorporating essential instruction and key educational technology design principles (Mayer 
and Moreno 2003). KTEK design features that specifically target the needs of students who 
struggle with math include: (a) personalized instruction, scaffolding, and feedback; (b) de- 
liberate practice to build fluency; (c) embedded formative assessment; and (d) customizable 
features allowing teachers to monitor and report findings (Baker et al. 2002). 

In each study, instructors were provided five color-coded class iPads with headphones, 
and students received their own unique passcode with which they could securely access the 
intervention system. Instructors were asked to support target students’ individual use of the 
KTEK intervention in small groups of five students for 15 min per day, 3 days per week, 
defined here as the duration of use target, in addition to regular, whole-class mathematics 
instruction. To increase attention and engagement, each 15-min intervention session 
included at least two separate learning activities. These use recommendations were devel- 
oped with a consideration for the specific developmental characteristics of the target pop- 
ulation and based on previous usability and feasibility tests of intervention session length. 

Preliminary analyses of pilot study data indicated that KTEK students experienced 
substantively important, but not statistically significant positive effects (What Works 
Clearinghouse 2008) in both number line estimation (Hedge’s g = .36) and magnitude 
comparison (g = .36), as compared to their control peers (Shanley et al. 2015). Further- 
more, KTEK students who experienced success in the intervention and mastered at least 
75 % of the activities they encountered demonstrated additional gains in missing number 
identification, oral counting, and two distal measures of number sense and early mathe- 
matics achievement (Strand Cary et al. 2014). Additionally, KTEK was generally well 
received by participating students; 68 % reported that they would “play KTEK at home if 
it was available” and the average likability rating was 1.6 on a scale from 1 to 5, with 
lower scores being more desirable. 
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Research design 


A convergent parallel mixed-methods design (Creswell 2013) was used to evaluate the 
relationships between instructor experience with technology and the implementation of 
KTEK in authentic settings. In both studies, quantitative data (i.e. KTEK analytics) and 
qualitative data (i.e., instructor surveys and interviews) were collected and analyzed sep- 
arately. Next, findings from analyses of the parallel research constructs were used in 
concert to address the research questions and the various types of data were examined for 
convergence and consistency. For example, instructor experience categorizations were 
verified with interview evidence and instructor reports of technology experience and 
intervention implementation were cross-tabulated with targeted KTEK analytics to explore 
relationships between experience and implementation. 

This mixed methods research was conducted within the context of two distinct devel- 
opment studies with unique emphases. Focal concerns for Study 1 included whether 
(a) KTEK functioned as intended, (b) participants enjoyed using KTEK, (c) the inter- 
vention could be integrated feasibly into the school day, and (d) the apps logged the desired 
data accurately. Results from Study 1 were utilized as part of an iterative cycle of tech- 
nology-based intervention development that included initial user testing, the feasibility 
study, and a pilot study interwoven with ongoing development and refinement. Prior to 
implementing the pilot study, development work aimed to improve the stability of the 
KTEK system and training materials were revised to provide targeted support to instructors 
with minimal technological experience. Focal concerns for Study 2 included all of those 
from Study 1 with the addition of a primary research question that investigated the extent 
to which KTEK students demonstrated mathematics achievement gains as compared to 
control students who used a similarly structured iPad-based literacy intervention. 


Measures 


The nature of the KTEK development process allowed for an authentic mixed methods 
approach derived from a variety of data sources. Measures included KTEK analytics (i.e., 
student usage data tracked by the student app), and student and instructor feedback pro- 
vided through surveys and interviews. 


KTEK analytics 


Individual student usage data reflected the extent to which instructors adhered to the 
recommended schedule of use for each student (i.e., duration of use target defined as 
15 min sessions, at least three times per week). Average daily duration of use per student 
was calculated based on total minutes of use divided by days of use. Daily duration of use 
values were aggregated across morning and afternoon classes to generate the percentage of 
students meeting duration of use targets by teacher. Because EAs worked across a range of 
classes, this process was applied for EAs, as well. In some cases, the percentage of students 
meeting duration of use targets were aggregated across instructors who reported similar 
levels of comfort with technology and/or barriers to technology use. These percentages 
were calculated by (a) combining all students taught by an instructor who reported a certain 
type of barrier or level of technological experience, and then (b) dividing the total number 
of students who met the implementation target by the total number of students. 
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Ranges of durations of use were also calculated for each instructor to evaluate the extent 
to which there were consistent minutes of use among students in a given class. To calculate 
the consistency of student exposure to the intervention or variability in implementation, the 
average range of use for each instructor was determined by subtracting the minimum 
average daily duration from the maximum daily duration. This value reflected the vari- 
ability of student use within each instructor’s class. High ranges of use indicated that some 
students used the intervention for many minutes on each occasion of use, while their 
classmates used the intervention for far fewer minutes each day. Conversely, small ranges 
of use indicated that instructors implemented the intervention with greater consistency and 
that there was less variability in the number of minutes of use each day between students. 


Surveys and interviews 


Instructors completed brief online surveys and participated in informal interviews (pre- and 
post-intervention). Surveys and interviews were comprised of 12-30 questions aimed at 
exploring instructors’ technology expertise, use of technology in the classroom, level of 
satisfaction with the intervention, and observations of students’ experiences with the 
intervention. All instructors completed all interviews and the initial surveys; with two 
opting out of the exit survey. Researchers coded and categorized survey responses to 
identify instructor technology experience themes. 

Survey responses were compiled with teacher interviews to identify patterns of 
responses. This analytic step had two main aims: (1) form categories of levels of prior 
experience with iPad technology (i.e., low, medium, and high) and (2) condense pre-study 
technology-use barriers into two distinct categories representing both first and second- 
order barriers (Ertmer 1999). First-order barriers were logistic (e.g., there is not enough 
time to integrate technology) or tech-based (e.g., troubleshooting technological difficulties 
is too time consuming). Second-order barriers were internal to the instructors imple- 
menting the intervention (e.g., I am not comfortable with technology). 


Results 
Overall experience and implementation 
Study 1 


Instructors reported a range of pre-intervention experience with iPads. One instructor 
reported no prior experience using iPads and one instructor reported moderate levels of 
experience with some common iPad functions (e.g., downloading and managing apps, 
taking pictures, sending and receiving email, accessing the internet, etc...). The four 
remaining instructors reported high levels of comfort with the same common iPad func- 
tions. Additionally, five of six instructors reported at least one barrier to integrating edu- 
cational technology in their classrooms; one instructor reported no barriers to technology 
use. See Table 1 for a complete list of barriers by type. Across all classrooms, 42 % of 
students met duration of use targets (i.e., the primary measure of implementation fidelity 
equal to 3 days per week, 15 min per day); however, student use varied. In some class- 
rooms, nearly 60 % of student users met the duration of use target. In others, only 20 % of 
students met the target. 
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Table 1 Barriers surveyed by type 


First order Second order 
I have inadequate time to learn how to use I am not comfortable with technology 
technology 
There is inadequate training or help offered I am worried I will break the technology 
Technology can be unreliable I am worried that I will not be able to help the students 
troubleshoot 
Class periods are too short I am worried about expensive equipment in the classroom 


Hardware, software, and/or apps don’t work _ I find it difficult to find useful software or apps 
properly 


Study 2 


Study 2 instructors also reported a range of pre-intervention experience with iPad tech- 
nology. Two instructors reported no prior experience using iPads; five reported moderate 
levels of comfort with some common iPad functions. The six other instructors reported 
high levels of experience with iPad functions. Additionally, just over half of the instructors 
reported at least one barrier to integrating educational technology in their classrooms, 
while five instructors reported no barriers to technology use. Across all study 2 classrooms, 
87 % of students met duration of use targets, but there was still substantial variability in the 
percentage of students meeting use targets across classrooms. In two instructors’ groups, 
none of the student users met the duration of use target, while 100 % of the student users 
met the target for use in other study 2 instructors’ groups. 


Experience with iPads as an implementation factor 
Study 1 


The instructor with the least iPad experience provided the most consistent intervention 
exposure represented by the narrowest range of use among students (see Fig. 1). In con- 
trast, instructors with moderate experience demonstrated the least consistent (i.e., most 
variable range) intervention exposure among students despite pre-intervention reports of 
some experience with iPads. Similarly students of instructors with high levels of iPad 
experience received less consistent intervention exposure; however, high experience 
instructors had more students who met duration of use targets (see Fig. 2). Both low and 
medium experience instructors had far fewer students meeting the duration of use target. In 
other words, high experience instructors had more students meet the duration of use target, 
but a number of their students were far below that target, as well. In the less experienced 
instructors’ classes, there was less variability in minutes of use between students, but few 
students met the average duration of use target compared to students of the instructors with 
more iPad experience. 


Study 2 
Findings from Study 1 were largely confirmed in Study 2 for the instructors with ample 


iPad experience. That is, the high-experience instructors had slightly more variable use, but 
a high percentage of students who met the duration of use target. Study 2 instructors with 
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Fig. 1 Average within class variability of student use by teacher experience 
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Fig. 2 Percent of students meeting duration of use target by instructor experience 


little to no iPad experience were like their Study | counterparts in that their students 
received the most consistent exposure to the intervention (i.e., a small range of minimum to 
maximum average student use; see Fig. 1), but unlike the Study 1 instructors, the low- 
experience Study 2 instructors demonstrated the highest percentage of students who met 
the duration of use target. Potential explanations for these divergent findings are discussed 
below. Most notably though, medium-experience instructors again demonstrated the most 
variable implementation and also had the fewest students who met duration of use targets. 


Barriers to technology integration 
Study 1 
Prior to implementing KTEK, instructors reported a range of barriers to technology use 


including first-order logistic or tech-based challenges and second-order internal barriers. 
Figure 3 displays the percent of students meeting duration of use targets by the type of barrier 
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Fig. 4 Percent of students meeting duration of use target by number of barriers 


each instructor reported. Consistent with previous research (Ertmer 1999), second-order 
barriers appeared to have the most significant negative association with average duration of use 
implementation metrics. Specifically, instructors who reported second-order barriers to 
technology use had the fewest students meeting the duration of use target. Furthermore, a 
comparison of the number of barriers reported by each instructor with the percentage of 
students meeting the duration of use target represented by Fig. 4 reveals another potentially 
important relationship: Study 1 instructors who reported the fewest total barriers to classroom 
technology use had the highest percentage of students meeting duration of use targets. 


Study 2 


Study 2 instructors also reported both first- and second-order barriers to technology use. As 
evidenced by Fig. 3, second-order barriers again appeared to be most impactful on 
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implementation; however, instructors who reported zero barriers to integrating technology 
in the classroom had less desirable implementation as compared to instructors who 
reported simple first-order barriers. In fact, instructors who identified first-order barriers 
actually had the most optimal implementation of the three groups in Study 2. There were 
differences between Study 1 and Study 2 with regard to some of the relationships between 
the number of barriers reported by each instructor and the percentage of students meeting 
the duration of use target, as well (see Fig. 4). For example, instructors who reported one 
or two barriers to technology implementation had the most desirable intervention imple- 
mentation with 90-100 % of their students meeting use targets in Study 2. In fact, with the 
exception of the instructors who reported three or more barriers to technology imple- 
mentation, instructors who reported no barriers to technology integration had the fewest 
students meeting duration of use targets in Study 2. 


Discussion 


Given that instructors facilitate interactions between technology-based interventions and 
student users in busy classroom settings, the studies detailed here sought to investigate how 
instructor experience with iPad technology and barriers to technology integration impacted 
the implementation of an iPad-delivered kindergarten mathematics intervention. Analyses 
of intervention implementation by instructor technology experience found distinct—and 
sometimes counterintuitive—patterns in student intervention exposure. The patterns are 
summarized and discussed below. 

Instructor experience with iPads did not always align with consistent exposure to the 
intervention amongst students. In fact, instructors with the least prior iPad experience 
implemented the intervention with the least variability evidenced by the most consistent 
exposure to the intervention among students in both studies. However, whereas instructors 
with little prior iPad experience implemented the intervention most consistently, Study 1 
findings indicated that little iPad experience was associated with fewer students meeting 
intervention use targets. In Study 1, teachers with more pre-intervention iPad experience 
had higher percentages of students meeting implementation targets. In Study 2 though, the 
low-experience instructors demonstrated the highest percentage of students meeting 
duration of use targets. This improved relationship may be attributed to a combination of 
factors. Improved intervention system stability equated to fewer technological issues and 
instructors did not need to troubleshoot as often in Study 2. Additionally, instructors were 
given targeted training and professional development to support those with less iPad 
experience prior to Study 2. Finally, additional personnel (i.e., educational assistants) were 
employed to implement intervention sessions in the second study. 

In addition to instructor iPad experience, there are other issues to consider. Specifically, 
the quantity of barriers reported, in addition to the type or order of barrier, may be critical 
implementation factors. In both studies, instructors who reported numerous barriers to 
technology integration and those who reported second-order barriers to technology inte- 
gration demonstrated the lowest percentage of students meeting implementation targets. 
Notably, instructors who reported no barriers to technology integration also had less than 
optimal student use patterns (i.e., few students meeting implementation targets). These 
findings suggest professional development and ongoing teacher support aimed at both 
unconfident and perhaps somewhat overconfident instructors are particularly important 
components of technology-based intervention implementation programs. 
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Professional development and teacher support should have a two-fold purpose to ensure 
optimal technology-based intervention implementation. In addition to providing profes- 
sional development about the intervention’s most important features for affecting student 
learning (Ottenbreit-Leftwich et al. 2010), care should be taken to address teacher-per- 
ceived barriers to technology-based intervention use (Ertmer et al. 2012) and to provide 
ample support and experience to alleviate fears, misconceptions, and negative attitudes. 
Taken as a whole, the results from both Studies 1 and 2 suggest there is a need to support 
instructors who report some or moderate iPad experience, in addition to the naive 
instructors, to identify specific implementation support mechanisms. Intervention devel- 
opers and implementation support staff may also need to probe instructors who do not 
identify any barriers to technology implementation because it can be difficult to predict all 
potential barriers and there may be a tendency to oversimplify the task of implementing 
technology-based intervention. Troubleshooting guides and targeted trainings that prepare 
all instructors for potential pitfalls and provide specific strategies for overcoming seem- 
ingly insignificant barriers may be one way to support instructors who are implementing 
technology-based interventions. Lastly, as expected, instructors who report numerous 
barriers to technology use may often require more support than others to implement an 
intervention. In these cases, both first- and second-order barriers should be investigated, 
and systems of support should be initiated. 

Ongoing monitoring and support are also important to track changes in instructor buy-in 
and confidence with technology. Observations, practitioner notes, and regular, formal 
professional development allow for a more accurate picture of technology-based inter- 
vention implementation and timely support provision. For instance, in study 1, an 
instructor with no prior iPad experience found it quite easy to integrate the iPads and 
implement the intervention, while other instructors noted that their students required more 
support than they had expected. Instructors with minimal prior educational technology 
integration experience might report low levels of initial buy-in, but then easily integrate the 
technology-based intervention into their existing classroom structures. On the other hand, 
high-experience instructors might experience more difficulty integrating the intervention in 
their classroom structures and require more support than initially anticipated. This may be 
a result of intervention implementation helping to shape and create a classroom-based 
technology schema for instructors with less iPad experience, as opposed to more experi- 
enced instructors who may struggle to find a way to integrate new technology-based 
interventions into their existing technology-based classroom systems. Regular monitoring 
and professional development that is adaptive to individual teacher needs and beliefs 
(Hammonds et al. 2013) will ensure that instructors receive appropriate, timely, and 
effective support to develop effective technology-based classroom systems. 


Limitations 


The studies reported here navigated two kinds of investigative challenges: (a) Synthesizing 
raw student analytics to appropriately characterize student’s technology-based intervention 
experience and (b) measuring appropriate fidelity indicators of technology-based inter- 
vention implementation. First, students’ differential paces of learning and individualized 
intervention experiences can be difficult to capture and compare. Struggling students’ 
intervention experiences vary greatly, thus dosage variables (i.e., duration of use or pro- 
gress through the app) may require the incorporation of other relevant covariates to be 
utilized accurately in evaluations of implementation. In this study, total minutes of use 
were divided by total days of use to approximate the average duration of use for each 
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student. However, admittedly, various omitted first-order barrier variables may have 
affected each of these measurements. Additionally, time of exposure is only one small 
aspect of successful intervention implementation. 

Second, conducting detailed analyses of factors affecting intervention implementation 
requires multi-faceted data collection in authentic contexts. This translates to extensive 
researcher time and resources. The present studies utilized a relatively small and demo- 
graphically homogenous sample of instructors to evaluate how patterns of implementation 
were related to technological experience and barriers to technology integration. The results 
reported here suggest replicable patterns of behavior exist and knowledge of instructor 
experience with iPads and perceived barriers could inform professional development to 
support effective intervention implementation; however a larger, more diverse instructor 
sample is needed to improve the generalizability of these findings. 


Future research 


If technology is to fulfill its promise of providing the individualized instruction nec- 
essary to meet pressing academic challenges a number of key variables warrant further 
investigation. This study found variation in implementation based on key instructor 
variables including iPad experience and barriers to technology integration. Future 
research should focus on appropriately selecting and preparing accurate metrics of 
student use for analysis (from the vast array of analytic data recorded within any 
technology-based intervention) and recruiting a robust, diverse sample of instructors. It 
would also be helpful to actively incorporate implementation support and collaborate 
with instructors to identify specific needs, thereby impacting instructor implementation 
variables and subsequently, with greater fidelity of implementation, impacting student 
outcomes. 

Harnessing the potential of educational technology requires that instructors are 
trained on the intended uses and most important features of technology-based inter- 
ventions, and receive support to structure their classroom and use of interventions 
accordingly. Barriers to technology use are inevitable, thus it is essential to address 
specifically and mitigate first- and second-order barriers. This can be accomplished 
through preliminary and ongoing professional development and targeted implementa- 
tion support for instructors who experience numerous barriers to classroom technology 
use. Deliberate support and continued focus on resolving these challenges will support 
the further development and refinement of technology-based interventions intended to 
assist with the provision of individualized instruction necessary to ensure that all 
students reach their full potential. 
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